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Abstract — A parallel concatenated coding scheme consists of 
two simple constituent systematic encoders linked by an inter- 
leaver. The input bits to the first encoder are scrambled by the 
interleaver before entering the second encoder. The codeword 
of the parallel concatenated code consists of the input bits to 
the first encoder followed by the parity check bits of both 
encoders. This construction can be generalized to any number 
of constituent codes. Parallel concatenated schemes employing 
two convolutional codes as constituent codes, in connection with 
an iterative decoding algorithm of complexity comparable to 
that of the constituent codes, have been recently shown to yield 
remarkable coding gains close to theoretical limits. They have 
been named, and are known as, "turbo codes." We propose a 
method to evaluate an upper bound to the bit error probability 
of a parallel concatenated coding scheme averaged over all 
interleavers of a given length. The analytical bounding technique 
is then used to shed some tight on some crucial questions which 
have been floating around in the communications community 
since the proposal of turbo codes. 

index Terms — Turbo codes, concatenated codes, iterative de- 
coding. 



I. Introduction And Motivations 

KNOWLEDGE of the fact that increasing the codeword 
length 7t of block codes (or the constraint length of 
convolutional codes) leads to better performance dates back 
to Shannon theory. It is also well known that the complexity 
of maximum-likelihood (ML) decoding algorithms increases 
with n, up to a point where decoding becomes physically 
unrealizable. 

Thus the research in coding theory has seen many proposals 
aiming at constructing powerful codes with large equivalent 
block lengths structured so as to permit breaking the ML de- 
coding into simpler partial decoding steps. Iterated codes [1], 
product codes and their extension [2J, concatenated codes [3] 
and their generalized version [4], and large constraint-length 
convolutional codes with suboptimal decoding strategies, like 
sequential decoding, are nonexhaustive examples of these 
attempts, and some of them have been successfully employed 
in applications where large coding gains are required, such as, 
for example, deep-space communication. 
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The most recent successful attempt consists of the so-called 
"turbo codes'* [5|, whose astonishing performance has given 
rise to a large interest in the coding community. They are 
parallel concatenated codes (PCC) (see Fig. 1 for the case 
of block PCC) whose encoder is formed by two (or more) 
constituent systematic encoders joined through an interleaver. 
The input information bits feed the first encoder and, after 
having been interleaved by the interleaver, enter the second 
encoder. The codeword of the parallel concatenated code 
consists of the input bits to the first encoder followed by the 
parity check bits of both encoders. 

The suboptimal iterative decoder is modular, and consists of 
a number of equal component blocks formed by concatenating 
the decoders of the constituent codes (CC) separated by the 
same interleaver used in the encoder. Each decoder produces 
weighted soft decoding of the input sequence. By increasing 
the number of decoding modules, and thus the number of 
decoding iterations, bit error probabilities as low as 10*"° 
at Eb/No = 0.0 dB have been shown by simulation [6]. A 
version of turbo codes employing two eight-state convolutional 
codes as constituent codes, an interleaver of 32 x 32 bits 
and an iterative decoder performing two and a half iterations 
with a complexity of the order of nine times the ML Viterbi 
decoding of each constituent code is presently available on a 
chip yielding a measured bit error probability of 0.0 - 10~ 7 at 
E b /N 0 = 3dB [7]. 

Bandwidth-efficient versions of turbo codes, compared to 
trellis-coded modulation schemes have also been proposed [8], 
as well as turbo codes based on block (instead of convolu- 
tional) codes [9), [10]. 

A careful examination of the literature shows that, rather 
than being a sudden apparition, turbo codes are the result 
of a clever intuition building on several concepts already 
available. We can cite in general the literature on product 
and concatenated codes in relation with the idea of paral- 
lel concatenation, the pioneering work on symbol-by-symbol 
maximum a posteriori decoding of linear codes (1 1) and the 
proposals in [I2)-[14] of the soft-decisions Viterbi algorithm 
in relation to the way of implementing the iterative decoder. 
Very close to turbo codes are also the separable "filters" 
described in [15] to iteratively decode multidimensional codes. 

As for the applications, it must be mentioned that PCC's, 
like all codes with very long codewords, suffer from one 
important drawback, namely the delay due to the interleaver 
and to the iterative decoding (as an example, the previously 
mentioned "chip" has a latency of 2318 bits). This prevents 
them from being used in applications where the combination 
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of decoding delay and data rale leads lo intolerable delays, like 
digital telephony. A bcoad range of applications still remains, 
siu;h as digital audio and video broadcasting, data packet 
transmission, and space applications. It is also worthwhile 
mentioning that the interleaver inherently present in the PCC 
can prove beneficial for transmission on fading channels [8]. 

Since the successful proposal of turbo codes, neither a good 
theoretical explanation of the codes behavior/performance nor 
an adequate comprehension of the role and relative importance 
of the PCC ingredients (constituent codes and interleaver) have 
appeared. 

In terms of performance of PCC\s, apart from the measure- 
ments on the chip [7], what is known is essentially due to 
simulation |5], [8], [15]-[18|, which, in itself, is not at all a 
simple task, as it requires a huge amount of computer time to 
obtain reliable results down to bit error probabilities like 10~ G . 

As a consequence, a number of basic questions are still 
unanswered: 

1 ) What is the performance of the ML decoder ? 

2) What are the relative contributions of the constituent 
codes and of the interleaver length in determining the 
PCC performance ? 

3) For a given interleaver length, how sensitive is the 
performance to the interleaver choice ? 

4) How crucial is the use of recursive (feedback) systematic 
convolutional codes (as opposed to nonrecursive ones) 
as constituent codes of the PCC scheme ? 

5) How close are the proposed suboptimal iterative decod- 
ing algorithms to ML decoding? 

Answering these questions is certainly important from a the- 
oretical point of view. Some of them, however, have significant 
practical relevance as well. For example, questions 1 and 5 can 
encourage (or discourage) the search for improved decoding 
algorithms, and question 2 may lead to the optimization 
of the PCC for given system constraints such as delay or 
complexity. Question 3, in turn, is related to the importance of 
the interleaver optimization, a topic which has already received 
some * 4 cut-and-try" attention [ 1 71-| 19]. Finally, question 4 has 
been discussed in [20] where the authors seem to believe 
that recursive convolutional codes have superior merits in 
themselves, rather than only when used as CC of a PCC. 

Formidable complexity obstacles discourage classical the- 
oretical analysis of the PCC's. As an example, the code 
implemented in VLSI in |7], when seen as a whole convo- 
lutional code, consists of an equivalent time-varying convolu- 
tional code with 2 1030 slates, thus preventing any analytical 
evaluation of the main performance parameters. 

In this paper, we will try to shed some light on the theoret- 
ical comprehension of PCCs. We will propose answers to the 
previous questions, some of which may be only preliminary 
yet indicative of the right direction. 

In particular, we will define and evaluate an upper bound 
to the average performance of the ML soft decoder for a 
PCC, stemming from characteristics of the CCs. Owing to 
its definition, the average performance, expressed in terms of 
bit error probability, turns out to be independent from the 
particular interleaver used, and helps in assessing what can 



be gained with given CCs and with an interleaver of a given 
length. 

We will also present simulation results for PCCs with 
differently chosen interleaves of the same length and com- 
pare them with the proposed bound. The results show that 
"random" interleaves offer performance close lo the average 
ones evaluated through the upper bound, independent, to a 
large extent, from the particular interleaver. Bad interleaves 
are very easy to avoid in practice. 

Moreover, we will show that recursive convolutional codes, 
although providing almost the same performance as nonrecur- 
sive codes when used alone, are indeed crucial when embedded 
in a PCC as CCs. 

Finally, by comparing our bound on ML performance with 
simulation results based on iterative decoding, we will give 
heuristic evidence that the suboptimal algorithms can come 
very close to the optimum. 

To help the reader, we will accompany the description with 
frequent examples, and will start from the simpler case of 
PCC schemes using block codes as CCs (parallel concatenated 
block code (PCBC)) leaving for the final sections the more 
complicated case of parallel concatenated convolutional codes 
(PCCC). 

11. Notations and Definitions 

Notations and definitions will be introduced for the case of 
parallel concatenated block codes, the extension to convolu- 
tional codes being straightforward. 

Given an '(n.k) systematic block code C\ its well-known 
weight enumeraiing function (WEF) is 

n 

B C (H) = J2 B > ir 

where B, is the (integer) number of codewords with Hamming 
weight (number of ones) ? and H is a dummy variable. The 
WEF of a code can be used to compute the exact expression 
of the probability of undetected errors and an upper bound to 
ihe word error probability (21). 

We define the input-redundancy weight enumerating func- 
tion (1RWEF) of the code as 

A c \\\\Z) = J2 A «S yU ' ZJ 
«'•-./ 

where A w j denotes the (integer) number of codewords gen- 
erated by an input information word of Hamming weight w 
whose parity check bits have Hamming weight j. so that the 
overall Hamming weight is ir — j. 

The IRWEF makes explicit in each term of the WEF the 
separate contributions of the information and of the parity- 
check bits to the total Hamming weight of the codewords, and 
thus provides additional information on the (Hamming) weight 
profile of the code. It will prove crucial in the following when 
dealing with parallel concatenated codes (PCC), since the two 
input words to the constituent encoders, the- second being 
obtained by interleaving the first, share the same Hamming 
weight, so that the redundant bits generated by the two 
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encoders derive from terms of the [RWEF with the same input 
weight iv. 

We mention also that -the IRWEF characterizes the whole 
encoder, as it depends on both input information words and 
codewords, whereas the WEF only depends upon the code. As 
a consequence, the WEF is related to the word error probability 
of the code, whereas the IRWEF provides information on the 
bit error probability. 

Obviously, the following relationship holds true: 

B C (H) = A C (\Y - H. Z - H) 



with 



where 



A C (H.H) = Y, A "'J HW * J = Yl BkHk 



Example I: The (7.4) Hamming code has the following 
WEF: 

B C (H) = 1 + 7/7 3 + 7# 4 + H\ 

Splitting the contribution of the information and redundancy 
bits to the total codeword weight we obtain the IRWEF of 
the code 

A C (W. Z) = 1 + IV (3Z 2 + Z 3 ) + W 2 {3Z + 3Z 2 ) 

+ U /3 (1+3Z) + VV 4 Z 3 . (1) 



Consider now the conditional weight enumerating function 
A%(Z) of the parity check bits generated by the code C 
corresponding to the input words of weight w. It can be 
obtained from the IRWEF as 



\r=o 



so that we can also write the inverse relationship 

A C (W,Z) = £ W M '.42(Z). 



(2) 



Both IRWEF and the /l£(Z) can be used with the union bound 
to compute an upper bound to the bit error probability for ML 
soft decoding of the code over a channel with additive white 
Gaussian noise in the form 



k 
k 



0\Y 



ir=Z=e-** £ V- v o 



with 



D = V -4 



(3) 



(4) 



The second and third line of (3) represent two equivalent 
expressions to bound the bit error probability. The first expres- 
sion keeps distinct the contributions of information words with 
different weight w m whereas the second sums the contributions 
according to the overall weight m of the codeword through 
the coefficient D tn defined in (4). 

A tighter bound can also be obtained from (3) [22] exploit- 
ing the inequality 

erfc(vT+l/) < erfc( v / ^>~ y . 

It assumes the form 



A(e) 



, W ( j d min R c E b \ 
^2fc CrfC (V-AT-J 



jni^Ii dA C (W,Z) 

e A " aw 



(5) 



which admits of course a funher development like (3). 

For parallel concatenated convolutional codes the infor- 
mation and code sequences are semi- infinite and, as a a 
consequence, the summation (explicit in (3) and implicit in 
(5)) must be truncated to a finite value. For the bound in the 
second line of (3) the truncation involves the computation of 
the complete conditional weight distributions up to a given 
information weight, whereas for the bound in the third line the 
truncation leads to the computation of the weight multiplicities 
of the unconditional weight distribution up to a given overall 
weight of the code sequences. Computing algorithms and a 
comparison of the two approximations are discussed in [23 J. 

Using a finite number of terms in (5) transforms the upper 
bound into the approximation 



(6) 



In the following, all the results in terms of bit error probability 
will be computed using (6). 

Example 2: The conditional WEFs of the Hamming code 
(7,4) considered in the previous example are 

A%(Z) = 1 
Af (Z) = iZ 2 + 
A%{Z) = iZ + 3Z 2 
A%(Z) = l+3Z 
A${Z) = Z 3 

so that the upper bound on the bit error probability computed 
through (6) and (4) becomes 



Pb(e) < - erfc 



where ft c is the code rate. 
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C p (n,+n 2 -k,k) 










L - ~ * 




Systematic 
codeC j (n lf k) 




Interleaver 
length k 






Systematic ^ 
code C2 (n 2 ,k) 





the constituent codes 



Fig. 1. Parallel concatenated block code. 



III. Parallel Concatenated Block Codes 

Consider now a parallel concatenated block code (PCBC) 
obtained as in Fig. 1. Two linear systematic block codes C\ 
with parameters (7/1, k) and Co with parameters (ri2,k) 9 the 
constituent codes (CC), having in common the length k of the 
input information bits, are linked through an interleaver so that 
the information part of the second codeword is just a permuted 
version of the first one. The PCBC codeword is then formed 
by adding to the input bits the parity-check bits generated by 
the first and second encoder. The PCBC, that we denote as 
Cp* is then a (nj + /i 2 - k. k) linear code as the interleaver 
performs a linear operation on the input bits. 

If w is the (Hamming) weight of the input word, and z\ 
and Z2 the weights of the parity check bits introduced by 
the first and second encoders, respectively, the weight of the 
corresponding codeword of Cp will be w + z\ + z^. 

We want now 10 obtain the IRWEF ,4 C,> (IV. Z) of Cp start- 
ing from the knowledge of those of the constituent codes. For 
a given interleaver, this operation is exceedingly complicated, 
as the redundant bits generated by the second encoder will not 
only depend on the weight of the input word, but also on how 
its bits have been permuted by the interleaver. The only viable 
solution, in theory, would be an exhaustive enumeration of all 
possible cases; in practice, this is an impossible achievement 
for large A\ and this was precisely the reason for lengthy 
computer simulations. 

To overcome this difficulty, we introduce an abstract inter- 
leaver called uniform interleaver. defined as follows. 

Definition I: A uniform interleaver of length k is a prob- 
abilistic device which maps a given input word of weight w 

into all distinct I " j permutations of it with equal probability 

"(•)■ 

From the definition, it is apparent that the conditional weight 
enumerating function A^-(Z) of the second code becomes 
independent from that of the first code thanks to the uniform 
randomization produced by the interleaver. 

As a nice consequence of this, we can easily evaluate the 
conditional weight enumerating function of the PCBC which 
uses the uniform interleaver as the product, suitably normal- 
ized, of the two conditional weight enumerating functions of 



C) 



(7) 



Also, from (2) we obtain the IRWEF of the code C r as 

K 

A C '(\V. Z)=^ W"'AZ''(Z). (8) 

(/■=! 

Example 3: The IRWEF of the PCBC constructed using 
as constituent codes two identical (7.4) Hamming codes can 
be obtained plugging the conditional WEF obtained in the 
previous example into (7) and applying (8) 

A C ''(\V,Z) = 1 + W{2.2bZ 4 + 1.5Z 5 + 0.25Z G )+ 

+ W' 2 (1.5Z 2 + 3Z 3 + 1.5Z 4 ) + 

+ U r3 (0.25 + I.oZ + 2.25Z 2 ) + W 4 Z°. (9) 

Notice in (9) the presence of fractional coefficients represent- 
ing the multiplicity of the various terms. They are a direct 
consequence of the use of the uniform interleaver. o 

The introduction of the uniform interleaver permits an easy 
derivation of the weight enumerating functions of the PCBC. 
However, in practice, one is confronted with deterministic 
interleaves, which give rise 10 one particular permutation of 
the input bits. So, what is the significance of the preceding 
definitions and equations ? 

To answer this question, we prove now the main property 
of a PCBC which uses the uniform interleaver. 

Theorem I: Let A Cr *{\\\?) be the IRWEF of the code 
Cp K obtained using the particular interleaver Then 

E k [A c ''*{\\\Zj\ = A C *:(\\\Z) (10) 

where E/». means expectation with respect to the whole class 
of mtcrleavers. y 
Proof of Theorem I: The proof makes use of (2) through 
the following equality chain: 



E K .[A C 'U\WZ)] = E* 



(ID 



= 53 W"-E* [.■£''«• (Z)j 

w 

= ^ir u .l£''(Z) = A C '{W.Z). (13) 



(12) 



where the third equality comes from the definition of the 
uniform interleaver. QED 

A second result, which comes as a corollary of the previous 
one from the linear dependency of (6) with respect to the 
conditional weight enumerating function, is the following. 

Corollary /: The upper bound computed using the IRWEF 
A Cr (\\\Z) coincides with the average of the upper bounds 
obtainable with the whole class of deterministic interleaves. 

V 

The corollary guarantees that, for each value of the signal- 
to-noise ratio, the performance obtained with the uniform inter- 
leaver arc achievable by at least one deterministic interleaver. 

Example 4: We can check the result (10) by computing, for 
the simple example of Hamming code previously examined, 
the IRWEF s of the PCBC's constructed usine all the inter- 
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TABLE I 

fKWF.F CJF Till: P.A K.M.LI:!. CONCATENATED CODES BASED ON THE (7, -1) lf\MMINC CODE H)K ALL POSSIBLE IVTEKLE.WERS 



"4*crm. 


J 4 C >*(H'\Z) 


0123 , 
0321 
1023 
1320 


I + W(3Z 4 + Z 6 ) + W 2 (ZZ- + 3Z 1 ) + H' 3 (l + 3Z r ) + W 4 Z* 


3021 
3120 , 




0132 
0213 
0231 
0312 
1032 
1203 
1230 
1302 
2013 

£UJ 1 


> I + W(2Z 4 + 2Z 5 ) + W 2 (Z 2 + 4Z 3 + Z<) + W\2Z + 2Z 7 ) + W A 2? 


2103 
2130 
2301 
2310 
3012 
3102 
3201 
:t210 





io-' 
io- 2 
io- 3 

IO 5 
10" 6 
IO" 7 
IO" 8 

0 l 2 3 4 5 6 7 8 9 10 

Fig. 2. Upper bounds for Example 4. 

leavers originating from the 24 = 4! permutations of the input 
bits. The computed IRWEF's are reported in Table I. 

From the table, it is apparent that, for this scheme, only two 
types of IRWEF arc possible: 

,1°''. (ir. Z) = 1 + \V(2Z 4 + 2Z 3 ) + U' 2 (Z 2 + 4Z 3 + Z 4 ) 
+ H' :I (2Z + 2Z 2 ) + U" 4 Z G (14) 

which derives from 18 different permutations and 

A Cr *{\\\Z) = i + _ir(3Z 4 + z G ) + ir 2 (3Z 2 + 3Z 4 ) 

+ ir 3 (l + 3Z 2 ) + W*Z* (15) 




Et/N 0 



which appears six times. The average computed over all 
possible interleaves yields 

A Cl '(\V.Z) = ^A Cr *i\WZ) + ^A r ^(\\\Z) 

which coincides with (9). 

The upper bounds obtained substituting (14). (15). and (9) 
into (6) arc plotted in Fig. 2. o 
Let n.k.t) denote the parameters of a /-error correcting 
(n. k) code. We have also analyzed the performance achieved 
by a PCBC using as CC the (63.57,3) and the (G3. 51.5) 
BCH codes. The interleaves have lengths A: = 57 and 51, 
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Fig. 3. Upper bounds to the bii error probability of two PCBC's using the (G3.57, 3) and (G3.51.5) BCH codes as CC's and interleaves til" lengths 
07 and 51. respectively. 



respectively. The results are reported in Fig. 3, where we 
also plot for comparison the results pertaining to the (7,4) 
Hamming code of the previous example. In this case, the 
performance of the PCBC schemes improves over that of the 
CC's, and a gain of 1 dB is obtainable. 

So far, we have used an interleaver with length k equal to 
that of the input word of the block code. It is straightforward to 
extend all previous results to the more general case where we 
use as basic codeword for a PCBC / consecutive codewords of 
the constituent code C\ as in Fig. 4. In particular, the IRWEF 
of the new constituent (Ini.fk) code C[ is given by 



A C *{\\\Z) = [A C *(\Y.Z)] 1 



(16) 



and similarly for the second constituent code C l 2 . 

The conditional weight enumerating function of the new 
component code can still be obtained from the IRWEF through 

^!(ir 



A$(Z) = — ■ 



1 0" A Q 



.Z) 



(17) 



From the conditional weight enumerating functions of the 
two new constituent codes, owing to the property of the 
uniform interleaver of length Ik, we obtain the conditional 
weight enumerating function of the (/(;/] +t/ 2 - k)Jk) PCBC 

as 



AlHZ) 



C) 



(18) 



PCBC (l(n 1 +n 2 -k) t lk) 




Fig. 4. Parallel concatenated block code with interleaver length tk. 

From this point on, the performance of the new PCBC can be 
obtained as before through (6). 

It is important to note that the PCBC obtained in this way- 
is in some sense a generalization of the concept of product 
codes. In fact, a product code which does not transmit the 
parity checks on parity check bits can be obtained from the 
PCBC using a row-column interleaver of length A* = A- 2 . 
Product codes with iterative decoding have been considered 
in [10], 

Example 5: Taking as component code the Hamming code 
of previous examples we can build different PCBC of increas- 
ing complexity by simply grouping / consecutive codewords. 

The first coefficients D tn defined in (4) for the resulting 
PCBC and / = 1/2.3.4,5 are reported in Table II. The 
effect of longer interleaves is clearly apparent from the table. 
Namely, the multiplicity of the terms which dominate the 



JM;M:DI:TT<) AND MONTORSI: UNVEILING TLRBO CODES: RESULTS ON RARALLLL COUINC SC IIIiMliS 




2 3 4 5 6 
Et/N 0 



10 11 



Fig. 5. Upper bounds to the bii error probability referring lo Example 5. The different curves refer to interleaves of various lengths lk\ 



TABLE II 

Coi-:ipicii:nts D,„ f-or the Evaluation of ma Bit Error Probability of 
run PCBC Obtajnfd from tiil (7.4) Hamming Cook and Uniform 
Inii.ru ,vvi :ks Gkoi pino 1.2.3. 4."> Consecutive Inplt Words 



Rate 1/3 PCCC 



Hamming 
distance 


/ 


I 


2 


3 


4 


5 


3 


0.1875 


0.02678 


0.01022 


0.00535 


0.00328 


- 4 


1 .875 


0.48214 


0.26590 


0.18214 


0.13815 


5 


3.75 


1.44642 


L. 06363 


0.91071 


0.82894 


6 


1.125 


1.20535 


0.95259 


0.81597 


0.73103 




0.0625 


3.83928 


3.11412 


2.77902 


2.57720 


8 




9.96428 


6.63116 


5.44900 


4.82972 


9 




23.7053 


16.4537 


13.6749 


12.2146 


10 


1 


33.3928 


32.8558 


30.8499 


30.0122 


11 




21.5089 


51.5566 


52.5036 


52.2468 


12 




12.3214 


104.044 


113.328 


117.619 


13 




7.16071 


192.425 


219.463 


231.044 


It 




4.40178 


311.545 


41S.307 


463.480 


15 




6.26785 


379.400 


737.146 


894.473 


16 




1.23214 


316.259 


1207.75 


1620.00 


17 




0.04464 


227.272 


201S.09 


3016.75 


18 






148.853 


3120.65 


5357.44 


19 






95.6634 


4271.70 


9176.27 


20 




1 


74.2149 


4865.35 


14962.0 



performance (those with lower Hamming weight) decreases 
when / increases. Also. Hamming distances not present in 
the consiituent codes show up. The minimum distance of the 
PCBC is still 3. as for the constituent codes. 

Applying the upper bound (6) we obtain the results reported 
in Fig. 5. where the bit error probabilities for the considered 
code and various interlcaver lengths I = L2. 1G\20. 100 are 
plotted versus the signal- to- noise ratio Ei/Xq. For compari- 
son, the curve of the constituent code is also reported. 

The figure shows that a gain of 2 dB can be achieved 
increasing / at the expense of an increased delay. Also, we 
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Fig. 6. Encoder structure of ;» PCCC. 

notice from the results that the beneficial effect of increasing 
the interleaver length tends to decrease the larger / becomes, o 
The results obtained in the example illustrate an interesting 
phenomenon that will be observed again: performance im- 
provements for a significant range of signal-to-noise ratios 
can be obtained with PCC's without increasing tor almost 
independently from) the minimum distance of the CC's. We 
ought to mention that independence of bit error probability 
performance from the code minimum distance for codes with 
large codewords length was already foreseen in |24|. 

IV. Parallel Concatenated Convolutional Codes 

The first applications of parallel concatenated coding 
schemes used convolutional codes as constituent codes. The 
resulting codes have been named by the authors turbo codes. 
and the main reason for their successful implementation 
resides in the availability of efficient algorithms for soft 
iterative decoding |5|, [7 J. [25|. In our context, we will call 
them parallel concatenated convolutional codes (PCCC). A 
block diagram showing how they work is presented in Fig. 6. 
The behavior is similar to that of a PCBC. the main difference 
being the fact that now the interleaver of length X does not 
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l ; ig. 7. Hyper-trellis for ihc PCCC. 

contain an integer number of input words, since the input 
sequences are infinite in length. 1 The figure represents the 
case of a rate 1/3 PCCC obtained from two rate 1/2 CCs. 
Several generalizations are possible. The number of CCs can 
be more than 2. and their rates can be different. Also, the 
final rate of the PCCC- can be increased by puncturing the 
redundant bit sequences at the encoders outputs. 

Wc will break the performance analysis of PCCC into two 
steps, the tirst performing an exact analysis to obtain the upper 
bound to the bit error probability, and the second showing how 
to obtain an accurate approximation which drastically reduces 
the complexity analysis. 

.4. Exact Analysis 

Consider a PCCC formed by an interleaver of length N and 
two convolutional CCs C\ and Co whose trellises have mi 
and ni 2 states, respectively. 

To examine the full dynamic of the PCCC, we must consider 
a hyper-trellis with m x - m 2 states, like the one depicted in 
Fig. 7. 

The state 5,^ -of the hyper-trellis corresponds to the pair of 
stales .si, and s 2j for the first and second constituent codes, 
respectively. Each branch 5 U — S rti t in the hyper-trellis 
represents all paths which start from the pair of states su, 
s 2j and reach the pair .si ? „. s 2 i in N steps (see Fig. 8). 

Thus when embedded in a PCCC using an interleaver of 
length A 7 , the constituent convolutional codes contributions to 
the final codeword (or code sequence) derive from TV-truncated 
versions of their input information sequences, or, equivalently, 
from trellis paths of length .V. 

Let Af/ m ' wt (\V.Z) be the label of the branch S ti -> S ml 
of the hyper-trellis. It represents the 1RWEF of the equivalent 
parallel concatenated block code obtained by enumerating the 
weights of all A r -lruncated sequences of the PCCC joining 
the hyper- states S t ; and S m/ . Once we know all these labels, 
the performance of the PCCC can be obtained through the 
standard transfer function bound approach [221 applied to the 
hyper-trellis. 

1 This observation, and the computing algorithms explained in the following, 
refer to ihc case of continuous transmission and decoding. When the PCCC 
is used as a block code wiih trellis termination, the same procedure described 
I or J*CBC can be applied. 




Fig. 8. Associations of states and paths of the trellises of constituent codes 
to states and paths of the PCCC hyper-trellis. 

To derive the branch labels of the hyper-trellis we can use 
the same procedure as that applied in Section III to parallel 
concatenated block codes, as we have seen that each label is 
indeed the IRWEF of a particular equivalent block code with 
information word length equal to N } 

We start with the conditional weight enumerating functions 
Af£ (ti;: Z) 3 of the equivalent block codes associated with the 
constituent convolutional codes. These functions enumerate all 
possible paths connecting the state s with the slate n in A r 
steps for the kth constituent encoder, k = 1. 2, and can be 
obtained from the transfer functions of the CC\s as described 
in the Appendix. From them, we compute the conditional 
weight enumerating functions Afj' ml (w. Z) of the equivalent 
parallel concatenated block codes. Owing to the properties of 
the uniform interleaver, they are simply the normalized product 
of A*(w.Z) with Aff(w.Z) 

n Af*(w.Z)-A < i'f{w.Z) 
V" ) 

Then, we obtain the IRWEF from the corresponding condi- 
tional weight enumerating functions through (2) and, finally, 
use the transfer function approach to get an upper bound to 
the bit error probability. 

An example will clarify the whole procedure. 

Example 6: Consider the PCCC obtained by Unking two 
identical 2-state recursive convolutional constituent codes with 
an interleaver of length N = 1. The resulting encoder structure 
is depicted in Fig. 9. 

First, we need to derive, using the algorithm described 
in the Appendix, the four conditional WEFs that 
enumerate all the possible paths connecting in four steps the 

2 Actually, only the label .4 ( f Ml # ',,,,( \\\ Z) describes a linear code containing 
the all "0" word; the other labels refer lu easels of this code. This has no 
effect on the analysts. 

3 For clarity, we slightly changed the notation of the conditional weight 
enumerating function by inserting the conditioning weight u within the 
parentheses. 
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TABLE II! 

Conditional Wbioirr Enumerating Functions Enumhk \tint. All Possimj£ Paths Connecting 
Styik s, with State Sj in Four Steps for the Constituent Encoder or Example 6 



Ag(tp,Z) 




w = 0 


w = I 


w = 2 


u> = 3 


w = -1 


00 


1 




2Z 2 + Z 3 + 3Z 




z- 


01 




z + z 2 + z 3 + z 4 




2Z 2 + 2Z 3 




10 




1 + z + z 2 + z 3 




2Z + 2Z 2 




11 


z 4 




Z + 2Z 2 + 3Z 3 




z 2 



TABLE IV 

Conditional W'lk.ht Enumerating Functions Labeling the Hyper-Trellis Describing the Dynamic or the PCCC oh Example ft 









KZ) 






ij y ml 


w = 0 


w = 1 


w = 2 


If = 3 


IL> - 4 


00,00 
00,01 
00,10 
00,11 
01,00 
01,01 
01,10 
01,11 
10,00 


I 




9Z' + !2Z J -MOZ<+4Z* + Z° 




Z 4 


Z*+7Z>+*Z*+AZ>+ZZ*+7?+Z* 


6 


4Z*+8Z s -hz« 


z 4 


4 


3Z*+8Z 3 4- 1 4Z 4 +8Z* +3Z 6 


4 


Z 4 




Zf 2Z* +3Z*+4Z 4 +3Z* +2Z« +Z' 


6 


4Z 3 +8Z 4 + 4Z S 




4 




4 




10,01 




10, 10 




10,11 
11,00 
11.01 
11, 10 

11,11 




1+2Z+3Z* +4Z 3 +3Z« 4.2Z»+2» 




4Z 2 +«Z 3 +4Z* 




z 8 


4 


Z i +4Z 3 + X0Z*+12Z*+9Z*) 


4 


z 4 









Thus the average IRWEF /l r ' (U'.Z) of the PCCC can 
be found by applying the transfer function technique to this 
hyper-trcliis. * o 



B. An Accurate Approximation 

In the preceding example, as the encoder had a very simple 
structure and the interleaving length was only 4. an analytic 
approach could be used to determine the exact expression of 
the average performance of the scheme. 

In general, for long interleaves and codes with larger 
constraint lengths, the hyper-trellis is completely connected, so 
that the number of branches increases with the fourth power of 
the number of states (supposed to be equal) of the CC's. Thus 
although the complexity of the analysis is only related to the 
CC's and not to the interleaver length, it may become very 
large. Our experience shows that this is the case for CC's 
with more than eight states. 

To overcome this difficulty, we propose a much simpler 
analysis. It is based on approximating the complete transfer 
function of the hyper-trellis with the IRWEF A^\ m (\Y.Z) 
which labels the branch joining the zero states of the hyper- 
trellis. It describes all paths which diverge from the zero states 
of both CC's and remerge into the zero states after A r steps. 




Fi£. 9. Parallel concatenated convolulional encoder of Example 6. 



state s, with the state of the CC. The results are summarized 
in Table III and refer to both identical CC's. 

The previous results can be used to construct, through (19), 
the conditional weight enumerating functions Af- P mi (w % Z) 
and. through (2). the labeling IRWEF Af£ ml (\\\ Z) of the 
hyper-trellis (Table IV). 

The hyper-trellis describing the dynamics of this scheme 
is then a 4-state trellis. The technique that leads to the 
computation of the performance of this scheme is the same 
as for a classic time- invariant convolulional encoder. 
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0 1 2 3 4 5 6 7 

Fig. 10. Comparison between exact and approximate techniques to evaluate the performance of a PCCC. The results refer to the 2-siatc PCCC of Example 
6. with .Y = >. 10.1000. Curves labeled "a" refer to the approximation, and curves labeled "fc" to the exact analysis of the hyper-trellis. 

code with generators (5, 7), free distance 5, and encoder as 
shown in Fig. 11. We have constructed different PCCCs 
through interleaves of various lengths, and passed through 
the previous steps to evaluate their performance. 

Upper bounds to the error probability based on the union 
bound (the transfer function approach for convolutlonal codes) 
present a divergence at low values of signal-to-noise ratio. 
We have checked the influence on the performance bounds of 
truncating multiple error events of the convo!utional code on 
the basis of their weight. Fig. 1 2 shows the bit error probability 
of a PCCC using an interleaver of length 1024. for different 
truncation weights (25, 30, 155. 38). 

Since we did not notice a convergence of the curves, we 
extended the coefficients D nt defined in (4) by means of an 
extrapolation based on the exponential law 

D, n (:r) = exp(a 4- ;ix). 

where the parameters a and ;i have been obtained through a 
mean-square optimization over the known values of D m . The 
curve labeled "extended" in the figure has been obtained in 
this way. The Dm profile for this code is shown in Fig. 13. 
together with the curve obtained through extrapolation. The 
"artificial" decrease in the multiplicities of D 1tt for large m 
due to the truncation can be seen from the figure, together with 
the accuracy of the extrapolation which allows us to include 
the multiplicities for larger values of m. 

The accuracy of the extension procedure has been checked 
by simulation. For CC\s with no more than eight states, it 



{Eh 



Fig. II. -J-siatc recursive convolutional encoder. 



To check the accuracy of the approximation technique, we 
have used the exact and approximate analyses to estimate 
the performance of the PCCC of Example 6 with different 
interleaver lengths, namely A T = 2,10,1000. The results are 
reported in Fig. 10. For :V = 2, the approximate and exact 
curves are significantly different above 10~ 4 . They merge 
around 10" 2 for A r = 10, and are completely indistinguishable 
for A r = 1000. Actually, this behavior starts from N = 20. 

In general, we have seen that the approximate method gives 
accurate results when the interleaver length is significantly 
larger (say, ten times) than the CC memory. For this reason, 
since the results that follow refer to this situation, we will use 
the approximate analysis. 

Example 7: Consider a PCCC employing as constituent 
codes the same 4-state recursive systematic convolutional 
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Fig. 12. Influence of the truncation of multiple errors based on their Hamming weight on the performance bound. 



TABLE V 

Coefficients D m for the Evaluation of the Bit Error Probability of 
the PCCC of Example 7 with Interleaves Lengths 100. 1000.10000 



Hamming 
distance 


<V 


100 


1000 


10000 


8 


3.8900E-02 


3.9S81E-03 


3.9988E-04 


9 


7.6590E-02 


7.9605E-03 


7.9960E-04 


10 


0.1136 


1.1918E-02 


1.1991E-03 


11 


0.1508 


I.5861E-02 


1.5985E-03 


12 


0.1986 


1.9887E-02 


1.9987E-03 


13 


0.2756 


2.4188E-02 


2.4017E-03 


14 


0.4079 


2.9048E-02 


2.8102E-03 


15 


0.6292 


3.4S46E-02 


3.228 1E-03 


16 


1.197 


6.5768E-02 


6.0575E-03 


17 


2.359 


0.1457 


1.3697E-02 


18 


4.3S3 


0.2984 


2.8543E-02 


19 


7.599 


0.5472 


5.2989E-02 


20 


12.58 


0.9171 


8.9441 E-02 


21 


20.46 


1.437 


0.1403 


22 


33.31 


2.144 


0.2082 


23 


54.65 


3.090 


0.2957 


24 


91.23 


4.465 


0.4177 


25 


154.9 


6.716 


0.6133 


26 


265.5 


10.67 


0.9577 


27 


455.6 


17.65 


1.574 


28 


779.0 


29.61 


2.646 


29 


1327. 


49.31 


4.430 


30 


2257. 


80.57 


7.267 


31 


3842. 


128.6 


11.60 


32 


6556. 


201.3 


18.04 


33 


11221 


311.5 


27.57 


34 


19261 


481.2 


41.88 


35 


33143 


748.8 


63.94 



permits precise estimation of the bound divergence, and will 
be used in the following to obtain error probability bounds. 

We notice, in any case, that all curves merge at a bit error 
probability around 10~ 4 and stay together from there on. 

Let us consider now the performance of different PCCC\s 
when varying interleaver length. In Table V the coefficients 
Dm needed for the evaluation of the bit error probability of 
the resulting PCCC for A r = 1(H). 1000. 10000 are reported. 
The effect of longer interleaves is apparent from the table. 
Namely, the multiplicity of the terms which dominate the 
performance (those with lower Hamming weights) decreases, 
when N increases, approximately as 1/:V, As to the free 
distance, it has increased from 5 (the CCs) to 8. 

Finally, in Fig. 14 we present the bit error probabilities of 
different PCCC's employing the same CC with intcrleavcrs 
of different lengths and, for comparison, the curve of the 
CC. Gains beyond 4 dB are achievable. The curves in the 
figure have been extended down to very low values of bit 
error probability, to show the progressive narrowing of the 
gap between the curves with different interleaver lengths. This 
is due to the fact that the free distance of the PCCCs is the 
same, and thus the curve will eventually merge at high values 
of signal-to-noise ratio. The curves also show the decrease 
by a factor 10 of the bit error probability for a factor 10 
increase of the interleaver length, as anticipated from the table 
of coefficients (Table V). o 



V. Comparison Between analytical Upper Bounds on 
ML Average Performance and Simulation Results 

We deal here with questions 2. 3. and 5 raised in the 
Introduction. 
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hg. 13. Profile of the coefficients D„, of the code of Example 7 and its cxiranolation (A* = 100). 
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t it:. 14 Average upper bounds to the bit error probability for the PCCC of Example 7 with uniform interleaves of lengths 1(H). 1000. 100(10. 

A. The Role of Constituent Codes performance for a wide range of bit error probabilities, even 

We have seen previously that increasing the interleaver though asymptotically the performance of the PCCC with uni- 
length for given CC's lead to noticeable improvements in form interleaver arc independent from the interleaver length. 
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Fiy. 15. Average upper bounds to the bii error probability for a PCCC using as CCs two recursive convolution! encoders with 2.-1.6. and to slates and 
uniform interleaves of length A" = 100 (continuous curves) and .V = 1000 (dashed curves). 



Let us now consider a uniform interleaver with two lengths 
.V = 100 and ;V = 10(H) as the building block of PCCCs em- 
ploying CC\s of different constraint length. We will examine 
the case of rate 1/3 CCs with constraint lengths 2.3. 4. 5 and 
the following generating matrices: 



9 — 



4 — 



1. 



I 



I + D 



- 1 



[ ' \ + D + D 3 J 



1. 



1 -f D 4 



I + D' 2 + £>' + D 4 



For the case of constraint length 5 we have examined two 
codes: the first uses as denominator of the generating matrix 
a primitive polynomial, whereas the second (marked with the 
asterisk) is the one used in the simulation of [5]. 

Their performance is shown in Fig. 15. Continuous curves 
refer to the interleaver length 100, while dashed curves are for 
■\ =z 1000. A few comments seem appropriate. At low bit error 
probabilities, say 10~\ the coding gain yielded by increasing 
the complexity of the CC s is rather large, namely around 6 
dB passing from 2 stales to lb states for both length N = 100 
and .V = 1000. This can help in those situations where 



delay must be kept low. in the sense that interleaver length 
(and thus delay) can be traded with CCs complexity. As an 
example, the PCCC based on 8-state codes with N = 100 has 
better performance than the PCCC based on 1-state codes with 
AT = 1000, yet reduces the delay. Similar considerations can 
be developed for higher values of the bit error probability. A 
comparison between the two lo-state codes show thatihe code 
used in [5 J (curves marked with an asterisk) is sensibly worse 
than the one we propose employing a primitive polynomial. 

Apart from the effect of the constraint length of constituent 
codes , there are also important effects due to the choice of the 
CCs for a given constraint length. This "optimization" of the 
constituent codes is dealt with in a companion paper [26]. 

B. Maximum- Likelihood and Suhoptumd Iterative Decoding 

The upper bound (6) is know n to be light for values of the bit 
error probabilities lower than 10~ 3 — \0~ 4 . It represents then 
a good estimate of the average performance of the ML soft 
decoding of PCC On the other hand, the practical importance 
of turbo codes resides in the availability of a simple suboptimal 
iterative decoding algorithm. To compare its performance with 
those of ML decoding, we have simulated iterative decodi ng 
using a * 4 !og-map M soft-output algorithm 4 applied to the PCCC 
employing two 4-state CCs described in Example 7. The 
results are reported in Figs. 16 and 17, for A" = 100 and 

4 This algorithm, a variation on the theme of sou output algorithms like the 
MAP algorithm described by Bahl (111 and the SOVA algorithm of 1 14|. is 
described in (271. 
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Eb/No 



Fig. 16. Comparison between the average upper bound and simulation results for the 4-states rale 1/3 PCCC of Example 7 with interleaving length 
A* — 100. The dashed curve refers to the analytical bound, whereas continuous curves represent simulation results obtained with increasing number 
of iterations A'y = 1. 2.3,4,5. 



jV = 1000, respectively. In the figures, the dashed curves 
refer to the bound, whereas the continuous curves show the 
simulation results obtained with an increasing number of 
iterations, A r 7 = 1.2. 3, 4, 5 for A 7 = 100 and Nj = 1, • • , 10 
for N = 1000. 

For the simulations, we have used interleavers chosen 
at random from the set of all permutations. The results 
were almost independent from the interleaver choice for error 
probabilities down to 10 ~ 5 , whereas they started to slightly 
diverge for lower values, owing to the different values of 
the free distance yielded by different interleavers. This sheds 
some light on question 3 of the Introduction, showing that the 
choice of the interleaver is not critical, as long as one avoids 
obvious "bad" choices like the replicating interleaver (the one 
which replicates the input sequence), for medium-high values 
of the bit error probabilities. A more careful choice yielding 
higher free distances can pay off for lower values of the error 
probability. 

The results also show that the performance obtained with the 
uniform interleaver are indeed very close to those obtainable 
with a practical appropriately chosen interleaver. Also, the 
convergence of the simulated curves to the ML bound for 
increasing number of iterations gives a heuristic evidence of 
the asymptotic optimality of the iterative decoding procedure. 
This is particularly evident in Fig. 17, where the simulated 
curves for an increasing number of iterations progressively 
conform to the ML performance bound- 



Finally, it is confirmed that approaching the ML perfor- 
mance requires an increasing number of iterations for low 
values of the signal-to-noisc ratio; four iterations are enough 
at P b (c) = 10~ 5 - Kr G , whereas we need 8-10 at P ft (c) = 

i(r 3 - io~ 4 . 



VI. Recursive and Nonrecursive 
Constituent Encoders 

In this section, we deal with question 4 of the Introduction; 
namely, the role played by recursive systematic con volution a) 
encoders as constituent codes (CC) of the PCCC. Through 
the analytical upper-bounding technique, we will show that 
turbo codes do require recursive convolutional encoders to 
work properly, and that this is a distinctive feature of turbo 
codes, in the sense that, when considered alone, systematic re- 
cursive (SR) and systematic nonrccursive (SNR) convolutional 
encoders have very similar performance. 

Consider a rate 1/3 PCCC employing as CCs two rate 1 /2 
convolutional codes with constraint length 2 whose encoders 
are shown in Fig. 18. Both are systematic codes 5 , the first one 
(Fig. 18(a)) is recursive while the second (Fig. 18(b)) is not. 
They have the same transfer function and thus the same error 

5 Using systematic codes is not strictly required; howexcr. it simplifies ibe 
decoder and has no effects on attainable performance, so that we will limit 
ourselves to them. 
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Fig, !7. Comparison between ihe average upper bound and si mutation, results for the 4-states rate 1/3 PCCC of Example 7 with interleaving length 
V = 1000. The dashed curve refers io the analytical bound, whereas continuous curves represent simulation results obtained with increasing number 
of iterations \ f — 1. .10. 
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Fi«. IS. Two-sialc recursive (al and nonrccursivc (b) encoders. 

event probability.* The bit error probability depends on the 
input-output correspondence of the encoders, and thus is not 
the same, although the difference is small. Moreover, in this 
case, as Fig. 19 shows/' the systematic nonrecursive encoder 
behaves better than the recursive one for Eu/N Q larger than 
1 dB. 

We now take sequences of length 100 from both CC*s, 
and compute their conditional weight enumerating functions 
A^ n {u\ Z) for weights of the input sequences up to 10. The 
results are plotted in Figs. 20 and 21, for the recursive and 

h Actually, ihis simple case is one of the very few where we can find two 
equivalent codes, one SR and the other SNR. with the same number of states. 

' In |20| the authors claim ihai RS convoluiional encoders yield significantly 
tower error probabilities than SNR codes, even when they have the same 
transfer function, the key fact being the inapplicability of the Viterbi algorithm 
for decoding and of the transfer function bound for performance evaluation. 
To check their conclusions, we have obtained the curve of Fig. 19 using both 
the transfer function approach and an equivalent block code with very large 
hUnrk si/e. up to A* = It MM). 



nonrecursive CC, respectively. The differences are apparent. 
For each weight of the input sequence, the redundancy weights 
generated by the input sequences of the SR CC span a broad 
range with a rather uniform multiplicity, while, for the SNR 
CC the input sequences with small weight generate a small 
set of redundant weights of low value. These are responsible, 
after the convolution with themselves which leads to the labels 
of the branches of the hyper-trellis of the PCCC, for the poor 
behavior of the concatenated scheme, as will be seen soon. 
We notice also that, for SNR CC. information sequences w ith 
weight w = 1 generate error events of finite weight, while, for 
SR CC, error events start with w = 2. This will be proved to 
be crucial in Section VI- A. 

Using the exact bounding technique described in Section 
IV-B, based on the complete hyper-trellis of the PCCC, we 
have computed the bit error probabilities for the two types 
of CC's. The results are reported in Fig. 22 for both schemes 
employing the SNR code as well as the SR code as CC's. The 
curves show the bit error probability versus the signal-to-noise 
ratio for different interleaver lengths. 

Curve *VT* of Fig. 22 corresponds to the uncoded binary 
PSK which is reported for reference. Curves "ZT and "C* refer 
to the SNR CC: curve represents the performance of the 
PCCC obtained by simply duplicating the redundant bit of the 
CC, while curve "C derives from the use of an interleaver of 
length 1000. The results show that the differences with N are 
marginal (less than half a decibel), and limited to a short range 
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Fig. 19. Bit error probabilities for 2-state SR and SNR encoders. 
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Fig. 20. Coefficients of the conditional weight enumerating functions Aq 0 (u\Z) for the recursive encoder l .Y = 11)11). 
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Fig. 21. Coefficients of ihe conditional weight enumerating functions A% Q (w,Z) for the systematic nonrecursive encoder f.Y = 1UU). 



of error probabilities. In fact, the curves practically merge 
below 

A completely different behavior for the SR CC is offered 
by curves "Z>" and "£/' referring to the same situations as 
the previous curves "ZT and "C" Here, in fact, we notice a 
significant improvement for :V = 1000, yielding a gain of 3 
dB ai 10"°. Also in this case the curves will merge (being free- 
distance-independent from A' for the uniform interleaver), but 
this will happen at very low values of the bit error probabilities. 

Interestingly enough, for N = i (compare curves "ZT and 
"D'*) the SNR-based PCCC improves over the SR one. This is 
due to the fact that the same free distance of both rate 1/2 CC's 
(recursive and not) is obtained from different contributions of 
the input and redundant bits, so that duplicating the redundant 
bits leads to a larger free distance of the SNR-based PCCC. 
The hierarchy is completely reversed for N = 1000. 

A. An Approximate Analytical Explanation 

We have seen earlier that recursive constituent encoders 
do play a fundamental role in PCCCs, and that this is due 
to the interleaving gain they yield because of the uniform 
spread of the weight distributions corresponding to low-weight 
information sequences. We will give here a simple heuristic 
explanation of this fact, which illuminates the most important 
parameters of the CC's in determining the PCCC perfor- 
mance. s A more detailed and accurate analytical explanation 

*We ought io mention that Dave Forney came to the same conclusions 
shown hereafter after reading a first version of this manuscript, where they 
did not show up clearly. The line of thouyhi in this subsection follows closely 
his comments. 



can be found in [26], where the attention is focused on the 
optimal design of PCCC with respect to the constituent codes. 

Consider a PCCC with an interleaver of length ;Y and two 
identical convolutional constituent codes C. Let an error event 
of C have weight m = w + j, where w is the number of 
information bit errors and j the number of redundant bit errors 
and call w m \ n the minimum </• in any finite-weight error event, 
and N mm the number of error events with w mm information 
errors and lowest weight 

m min = miii[/r mm + j] 
j 

per unit time in C. 

The number of possible information error events of weight 
ttf m j n contained in an interleaver of length :V is 

C) 

V" : min / 

so that the probability under uniform interleaving that the 
information errors are associated with the lowest weight 
error event is 




where the last expression holds for large values of A r . 

Equation (20) shows that the interleaver gain in the error 
coefficient is proportional to A" 1 ""'""", and. consequently, that 
this parameter w min is indeed a key design parameter of the 
CC's. Now, it is easy to see that //',„;„ is equal to 2 for all 
recursive convolutional encoders, which yields the interleaver 
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Fig. 22. Bit error probabilities comparison between PCCC using SR and SNR CC's. 
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gain increase 1 /;V noted in Example 7, and that it is equal 
to 1 for nonrecursive encoders, which explains the results of 
Fig. 22 being almost independent from N.-ll is worthwhile 
mentioning that is also equal to 1 for block codes, which 
explains the lower gain obtained with PCBC's of Section II. 

To quote Dave Forney 9 'Turbo codes seem to turn the 
conventional design principles on their head; they make error 
coefficients more important than minimum distance !*' Indeed, 
the main CCs design parameters to optimize a PCCC for a 
given interleaver length are w mui and m mm , and the last can 
also be significantly larger than the free distance of the code. 

VII. Prospects of Research 

There are several important open questions and topics for 
future research, including those in the following list. 

1) We have shown through examples that the iterative 
soft-output decoding scheme employed by turbo codes 
approaches the ML performance bound for increasing 
number of iterations. An important theoretical question 
concerns the convergence of the suboptimal algorithm 
to the ML decoding. Does it converge ? Under what 
conditions ? 

2) We have shown in some cases that a reasonable choice 
of (he interleaver leads to performance close to the 
average, and that the interleaver choice is not critical, 
as opposed to its length, which is indeed the main 
reason for the good performance of PCC's. However, 
the interleaver plays a role in determining the free 
(or minimum) distance of the PCC and, consequently, 
the asymptotic performance of the code. Constructive 
algorithms to find "good" interleaves of a given length, 

" Private communication. 



as well as theoretical results on limits to the achievable 
free distance for a given interleaver length and CC's 
would be important, especially for PCCC's employing 
short imerleavers. A few, very preliminary steps in this 
directions have been taken in [ 17], [ 19], [25], [28] using 
"cut-and-try" approaches. 

3) For some applications, a short decoding delay is a must. 
In these cases, the best compromise between interleaver 
length and CC's complexity certainly deserves attention, 
as does the optimization of the CC's based on the new- 
principles outlined in Section VI-A. 

4) Several iterative decoding algorithms have already been 
proposed. A comparison of them, as well as the search 
for new ones aiming ai reducing the number of iterations 
(and consequently delay ) required would be an important 
achievement. 

5) We have proposed here a bounding technique valid for 
both block and convolutional PCC's. While simple and 
good iterative soft-decoding algorithms for PCCC have 
been proposed and implemented, the same is not true 
for PCBC's. A first solution for the particular case of 
product block codes has been proposed recently in [ IOJ. 
However, a complexity analysis is still missing, and 
other alternatives need to be explored. 

6) In certain applications, bandwidth efficiency is required, 
and the best compromise between coding gain and 
bandwidth efficiency found so far is trellis-coded mod- 
ulation. A pragmatic approach to joint turbo coding 
and modulation was presented in |8], with promising 
results. A satisfactory approach to overall optimization 
as available for TCM is nevertheless still to come. 

7) It is a common belief that the performance of PCC 
degrade significantly if one tries lo increase the rate. This 



BNSDOCID: <XP 850506A__L> 



BKNItDliTTO AND MONTORS1 UNVEILING TURBO COOES: RtSULTS ON PARALU : .L*a>OING SCHUMES 



4:7 



♦ conclusion is based on simulation of PCC employing 
punctured CCs. Other solutions should be tried, such 
as using higher rato-CCs. The analytical tools presented 
here give a way to analyze them. 
8) We already mentioned that the interleaver inherently 
present in the coding scheme might prove beneficial for 
channels affected by fading. Some results obtained by 
simulation in this direction are contained in [8[. Ex- 
tension of the analytical bounding technique presented 
here for AWGN channels to fading channels would be 
important for both analysis and design purposes. 

VIII. Conclusions 

We have proposed for the first time a method to evaluate 
the bit error probability of a parallel concatenated coding 
scheme independently from the interleaver used. Crucial was 
the introduction of a probabilistic interleaver called uniform 
interleaver which permits an easy derivation of the weight 
enumerating function of the parallel concatenated code starting 
from the weight enumerating functions of the constituent 
codes The two cases of parallel concatenated block codes and 
parallel concatenated convolutional codes were considered. 
This analytical bounding technique was then used extensively 
to clarity some relevant aspects of this interesting and promis- 
ing online technique. 

ArPliNDIX 

Evaluation of thi: Conditional Weight Enumerating 
Function A f ~(Z) oy thi: Equivalent Block Code 

In this Appendix we show how to compute the conditional 
weight enumerating function .l£,(u,\Z) which describes the 
equivalent block code obtained as the set of sequences of the 
constituent code (CC) C leading to the CC trellis from state 
s 9 to slate s n in A* steps. For simplicity, we will only show 
the derivation of A^ {i (tt\ Z). which will be called A^.(Z) for 
brevity. 

We consider then for each constituent convolutional code 
an equivalent block code whose trellis representation is the 
truncation at step iX of the trellis of the convolutional code, 
and whose codewords lead the trellis into the identity state at 
step /V. Our goal is to derive the 1RWEF of such a block code, 
starting from the knowledge of a suitably defined error events 
enumerating function of the convolutional code. 

With such a definition, the number of codewords of the 
equivalent block code can be very large, but not infinite as for 
the convolutional code, so thai a short closed-form expression 
for its IRWEF does not exist. For this reason, we will use an 
algorithmic approach that allows the evaluation of the most 
significant terms of the IRWEF. 

Let us consider Fig. 23. By our previous hypotheses on the 
block code which approximates the constituent convolutional 
code, any codeword belonging to the block code is obtainable 
by combining the set of error events of the convolutional code 
with suitable sequences of "0" so that the total lencth equals 
A r . 

As an example, a single error event of length / smaller than 
N produces all codewords with ;Y - / zeros positioned before 




N N 

Fig. 23. How to obiain codewords of the equivalent block code from error 
events of the convolutions! code. 



and after the error event. All these codewords share the same 
input and redundancy weight, so that they can be grouped 
together. The multiplicity K[l. 1] of the block codewords 
produced from this single error event in the IRWEF of the 
block code equals the number of partitions of A* - / into two 
numbers: 

A'lU]=( Ar -/ +1 )=A'-i + l. 

Similarly, a single pair of error events with total length 
I produces all the codewords with zeros before, after, and 
between the two error events. Thus the multiplicity A'[/. 2] of 
the codewords produced from this single pair in the IRWEF 
of the block code equals the number of partitions of A r - I 
into three numbers 

A'(U]=f- 2 i + 2 ). 

Proceeding this way. one can obtain the general expres- 
sion for the multiplicity of codewords produced by a single 
combination of n error events with total length / 

*„.„,, (-*-,!-). 

Let T c (W y Z. LAI) be the transfer function of the convo- 
lutional code which enumerates all paths in the trellis leaving 
the zero state at step L and remerging into the zero state 
at or before step A', with possible remerging into the zero 
state at other steps in between, subject to the constraint that, 
after remerging, they leave the zero state immediately at the 
successive step 10 

T C (W,Z,LM)= Y, ^n tM lV i Z J L m tt a (21) 

where Tij t „ lt „ is the number of paths in the trellis produced 
by an input sequence of weight /'. with weight of the redundant 
bits equal to j % length m. and // rcmergings with the zero state 
(and hence concatenating 7/ error events). 

As for the case of block codes, we define the conditional 
transfer function T^(Z, L.<>) as 

T^{Z,L,U) = Yi T u .j. m . n ZiL m ll". (22) 

j.tn.tt 

Passing now to the codewords of the equivalent block code, we 
notice that each path of length m and number of rcmergings n 

10 Examples of these concatenations of single error events were shown in 
Fig. 23. 
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belonging to T£(Z*L.Q) gives rise to K\m,n] codewords 
with the same input ar.d redundancy weights, so thai the 
conditional IRWEF A^\Z) of the equivalent block code can 
be obtained as 



with 

in ,ti 

An efficient algorithm able to compute the most significant 
terms 1 1 of the transfer function T C (\V, Z.LM) has been 
implemented, elaborating on the algorithm described in [29] to 
evaluate the transfer function of a convolutional code, and then 
yielding as output the conditional IRWEF of the equivalent 
block code. 
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